Learning-based Face Synthesis for Pose-Robust Recognition from Single Image

A major challenge for automatic face based identity inference is the shear magnitude of uncontrolled factors than can result in a considerable change of shape and appearance, such as expression, illumination and pose. The problem is complicated further when only one image per person is available for training. In this paper we present an approach that uses a data driven and computationally efficient 2D technique for the synthesis of high quality non-frontal faces from a single frontal input face, that be used for extending the training set for each person, enabling 2D face recognition algorithms to improve their performance when dealing with pose variations. We use offline AAM fitting to obtain the locations of landmark points in a given frontal face image. Once the locations of the landmark points from the gallery images have been extracted, we use a regression-based approach to generate synthetic images at various poses. We first learn the correspondence between the landmark points of the set of frontal images exhibiting arbitrary facial expressions and their corresponding non-frontal images at arbitrary pose. Once this learner has been trained, the synthetic image of any other unseen face can be generated by predicting the locations of the new landmark points, followed by warping of the texture from the original frontal image. We start by extracting 3 vectors: Normalisation, Centroid, and Point Vector from each of the m frontal and non-frontal training images. Normalisation Vector is a 1D vector containing information about the normalisation distances used to normalise the feature vectors.